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Method For The Extraction of 
Image Features Caused by 
Structure Light Using Template 

Information 

Background of Invention 

[0001] The present invention relates to a system for observing objects in three 

dimensional space using structured light. More particularly, the invention relates to a 
multi-camera, three dimensional sensing system for providing non-contact gauge 
measurements of an object using known object templates and known epi-polar 
geometry to identify observable projections of structured light on the surface of an 
object, and to reconstruct corrupted portions of the projected structured light. The 
resulting set of data points is then compared with a set of predetermined ideal shape 
information data points, or a reference model, to identify gauge measurements of the 
object. 

[0002] Measurement systems such as laser range finders, illuminate an object 

undergoing measurement using structured light. Reflections of the structured light 
projected on the surface of the object are captured by two or more calibrated 
cameras, generating images of the illuminated portions of the object's surface. In 
some applications, the structured light is in the form of a set of laser planes. Where 
the laser planes intersect the surface of the object, a striping effect is achieved. By 
detecting these laser light stripes in the images of the object's surface, point 
correspondences can be established and triangulation techniques employed to 
reconstruct a representation of the surface of the object 

[0003] 

First, a single pixel of each stripe in an images from a first camera is selected. 
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Given that the position of the first camera lens in space is known and the selected 
pixel is known, then it is known that the point corresponding to the selected pixel lies 
on a known line drawn from the lens center out into space. This line appears as a line 
in an image from a second camera. This line is called the epi-polar line of the selected 
point in the first image. Since the position of the lens center of the second camera is 
also known, this epi-polar line can be calculated and drawn in the second image. The 
epi-polar line, when drawn in the second image, will intersect at least one, and most 
likely several, of the stripes of the second video image. It is known that one of the 
pixels where the epi-polar line and a stripe intersect represents the selected point in 
the first image. The actual coordinate location in space of the point corresponding to 
any of these intersection points is determined by simple triangulation. Since the 
position in space of each plane of light which created the stripes is also known, the 
single point of all the intersection points which correspond to the selected point in the 
first image is ascertained by determining the three-dimensional coordinate of each 
intersection point to determine if it lies on one of the known planes of light. The 
intersection point which lies closest to a known plane of light is taken as the selected 
point. 

[0004] The surface of many objects includes regions which are shiny or have otherwise 
poor reflectivity characteristics. Structured light projected onto these surfaces can 
result in multiple reflections dr clutter which, in turn, results in poor laser stripe 
identification in the resulting images obtained by the cameras. Furthermore, in some 
situations, entire image regions which are representative of portions of an objects 
surface may become corrupted due to noise or interference. 

[0005] Accordingly, there is a need for a method to enhance laser stripe identification in 
images, and for a method of reconstructing laser stripes in those portions of images 
which are heavily corrupted due to noise or interference. 

Summary of Invention 

[0006] 

Briefly stated, the present invention sets forth a method for generating a template 
guide representative of the surface of an object, and for utilizing the template guide 
to improve laser stripe signal to noise ratios and to compensate for corrupted image 
regions, thereby improving the accuracy of current laser range finding and 
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measurement systems. 



[0007] The foregoing and other objects, features, and advantages of the invention as well 
as presently preferred embodiments thereof will become more apparent from the 
reading of the following description in connection with the accompanying drawings. 

Brief Description of Drawings 

[0008] In the accompanying drawings which form part of the specification: 

[0009] Figure 1 is a simplified view of a image of laser stripes projected on a surface of 
an object, including both coherent and corrupted regions; 

[001 0] Figure 2 is a simplified diagram of a tangential flow field for a portion of the 
image shown in Fig. 1 ; 

||! [001 1] Figure 3 is a simplified diagram of a perpendicular flow field for a portion of the 
Jvt image shown in Fig. 1 ; 

w 

p [001 2] Figure 4 is a simplified view of a single laser stripe from an uncorrupted image 
region; 

111 [001 3] Figure 5 is a simplified view of a single laser stripe from a corrupted image region; 

In and 

[0014] Figure 6 is a collection of three images of a single laser stripe projected on a 
surface of an object, as seen from three individual cameras. 

[001 5] Corresponding reference numerals indicate corresponding parts throughout the 
several figures of the drawings. 

Detailed Description 

[001 6] The following detailed description illustrates the invention by way of example and 
not by way of limitation. The description clearly enables one skilled in the art to make 
and use the invention, describes several embodiments, adaptations, variations, 
alternatives, and uses of the invention, including what is presently believed to be the 
best mode of carrying out the invention. 
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[0017] 



If repeated measurements of objects of the same type are performed, a set of 



canonical images can be generated so that the laser stripes in each image can be 
identified manually. This process results in a set of templates which are representative 
of the surface of similar objects. Subsequent scans of similar objects will have similar 
laser striping patterns, with any deviations due to variations in the object surface 
morphology. The template structures represent prior knowledge of the surface of the 
objects, such that features in subsequent images which are inconsistent with the 
expected results, as represented by the templates, can be ignored or de-emphasized. 

[001 8] Templates can be used in various ways to increase the fidelity of the laser stripe 
localization process. By using the templates as a guide, a two-dimensional locally 
matched filter may be generated for each point in an image 1 0 of projected laser 
stripes 1 2A-1 2H on the surface of an object 1 3, such as shown in Figure 1 . Next, a 
flow field is established which defines an orientation for each point in an image. A 
flow field may be either a tangential flow field, as seen in Figure 2, or a perpendicular 
flow field, as seen in Figure 3. Each arrow shown in Figs. 2 and 3 represents the 
assigned flow for a given pixel comprising the image. 

[001 9] The flow field is generated using the templates by way of an orientation diffusion 
processes, such as, but not limited to, interpolation methods and relaxation methods. 
The filtering is done in two passes. In the first pass, each pixel (i,j) in an image 1 0 is 
given the value: 



where R is a curve which emanates from pixel (i j) and is always tangential to the flow 
field, r is a measure of arc length along curve R, and image(r) is the image intensity 
value for a point on curve R. The gaussian term localizes this one dimensional filter. 




R 



[0020] 



In the second pass, each pixel (ij) is given the value: 



*fc j) = X ( V W X gaussian(p)) 



p 



where P is a curve emanating from pixel (i j) and is always perpendicular to the flow 
field, and p is a measure of arc length along curve P. The result of this two pass 
approach is a two-dimensional local matched filter responsive to the original image 
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10. The matched filtering enhances much of the true signal while suppressing 
unwanted noise. 

[0021] Once an image 1 0 has been processed with the filters, non-maximal suppression 
techniques are utilized to identify the center of each laser stripe 1 2A-1 2H. In one 
embodiment, each raster line in an image 1 0 is scanned to identify points where t(i,j) 
is a local maximum with respect to the raster line. These points represent the 
detected laser stripe structure. In this way, the laser stripe signal to noise ratio is 
increased, resulting in an increase in measurement accuracy. 

[0022] Alternatively, a single pass approach could also be used by employing a single 

two-dimensional filter. Or, separable one-dimensional filters which are non-gaussian 
could also be employed within the scope of the invention. 

[0023] Due to problems associated with reflections and laser specularity, a region or laser 
stripe in image 10 may become corrupted. For example, as seen in Figure 1 , laser 
stripe 12E is corrupted and discontinuous. To determine if a local region of an image 
1 0 is corrupted by either reflections or laser specularity, pixels (ij) of the image 10 
which are representative of local maxima, or zero-crossings, are identified. If pixels 
(ij) representative of local maxima in a region are in a structured pattern, i.e. form a 
line or are "chained", such as seen in laser stripes 12A-12D, 12F-12H, and Figure 4, 
the region is considered to be coherent and not corrupted. Alternatively, if there are 
no pixels (ij) representative of local maxima in a region, or if the representative pixels 
(ij) do not form a structured pattern, i.e., are incoherent as seen in laser stripe 12E 
and Figure 5, the region is marked as corrupted. 

[0024] Using the generated template structure and known epi-polar geometry, the 

corresponding laser stripes in non-corrupted regions of the image 10 can be readily 
identified. Next, as seen in Figure 6, by using triangulation and projection techniques 
from multiple view of the object 1 3, the one or more laser stripes such as 1 2E in the 
corrupted regions of the image 10 can be synthesized by using corresponding 
uncorrupted laser lines in different images 14 and 16. 

[0025] 

In view of the above, it will be seen that the several objects of the invention are 
achieved and other advantageous results are obtained. As various changes could be 
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made in the above constructions without departing from the scope of the invention, it 
is intended that all matter contained in the above description or shown in the 
accompanying drawings shall be interpreted as illustrative and not in a limiting sense. 
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